Motion tracking, also known as Motion capture is the process of recording a live motion event and translating it into usable mathematical terms by tracking a number of key points in space over time and combining them to obtain a single three-dimensional representation of the performance. In brief, it is the technology that enables the process of translating a live performance into a digital performance. The captured subject could be anything that exists in the real world and has motion; the key points are the areas that best represent the motion of the subject""s different moving parts. These points should help resolve pivot points or connections between rigid parts of the subject. For a human, for example, some of the key points are the joints that act as pivot points and connections for the bones. The location of each of these points is identified by one or more sensors, markers, or potentiometers that are placed on the subject and that serve, in one way or another, as conduits of information to the main collection device.
There are a number of existing systems that can track the motion of human, animal or inanimate subjects. Existing motion capture systems are classified as outside-in, inside-out, and inside-in systems. These names are indicative of where the capture sources and sensors are placed.
An outside-in system uses external sensors to collect data from sources placed on the body. Examples of such systems are camera-based tracking devices, in which the cameras are the sensors and the reflective markers are the sources.
Inside-out systems have sensors placed on the body that collect external sources. Electromagnetic systems, whose sensors move in an externally generated electromagnetic field, are examples of inside-out systems. Inside-in systems have their sources and sensors placed on the body. Examples of these devices are electromechanical suits, in which the sensors are potentiometers or powered goniometers and the sources are the actual joints inside the body.
The principal technologies used today that represent these categories are optical, electromagnetic, and electromechanical human tracking systems.
Optical Motion Capture Systems
Optical motion capture is a very accurate method of capturing certain motions when using a state-of-the-art system. It is a real-time process with certain limitations, such as marker count and number of performers and cameras.
A typical optical motion capture system is based on a single computer that controls the input of several digital CCD (charge-coupled device) cameras. CCDs are light-sensitive devices that use an array of photoelectric cells (also called pixels) to capture light, and then measure the intensity of the light for each of the cells, creating a digital representation of the image. A CCD camera contains an array of pixels that can vary in resolution from as low as 128xc3x97128 to as high as 4096xc3x974096 or even greater.
Obviously, the higher the resolution, the better, but there are other trade-offs. The samples-per-second rate, or frame rate, has to be fast enough for capturing the nuances of very fast motions. By today""s standards, a CCD camera with a resolution of 4096xc3x974096 would be able to produce less than one frame per second. Another important feature is shutter synchronization, by which the camera""s shutter speed can be synchronized with external sources, such as the light-emitting diodes (LEDs) with which optical motion capture cameras are usually outfitted.
The number of cameras employed is usually no less than 4 and no more than 32, and they capture the position of reflective markers at speeds anywhere between 30 and 1000 samples per second. The cameras are normally fitted with their own light sources that create a directional reflection from the markers, which are generally spheres covered with a material such as Scotch-Brite tape. Infrared light sources are preferred because they create less visual distortion for the user. The marker spheres can vary from a few millimeters in diameter, for small-area captures, to a couple of inches.
The optical system must be calibrated by having all the cameras track an object with known dimensions that the software can recognize, such as a cube or a wand with reflective markers. By combining the views from all cameras with the known dimensions of the object, the exact position of each camera in space can be calculated. If a camera is bumped even slightly, a new calibration must be performed. It is a good idea to recalibrate the system after every few minutes of capture, since any kind of motion or vibration can shift the position of a camera, especially if the studio is located on unstable ground.
At least two views are needed to track a single point""s three-dimensional position, and extra cameras are necessary to maintain a direct line of sight from at least two cameras to every marker. That doesn""t mean that more cameras are better, because each additional camera increases post-processing time.
Once the camera views are digitized into the computer, it is time for the post-processing to begin. The first step is for the software to try to produce a clean playback of only the markers. Different image processing methods are used to minimize the noise and isolate the markers, separating them from the rest of the environment. The most basic approach is to separate all the groups of pixels that exceed a predetermined luminosity threshold. If the software is intelligent enough, it will use adjacent frames to help solve any particular frame. The system operator has control over many variables that will help in this process, such as specifying the minimum and maximum lines expected per marker so the software can ignore anything smaller or bigger than these values.
The second step is to determine the two-dimensional coordinates of each marker for each camera view. This data will later be used in combination with the camera coordinates and the rest of the camera views to obtain the three-dimensional coordinates of each marker.
The third step is to actually identify each marker throughout a sequence. This stage requires the most operator assistance, since the initial assignment of each marker has to be recorded manually. After this assignment, the software tries to resolve the rest of the sequence until it loses track of a marker due to occlusion or crossover, at which point the operator must reassign the markers in question and continue the computation. This process continues until the whole sequence is resolved and a file containing positional data for all markers is saved.
The file produced by this process contains a sequence of marker global positions over time, which means that only each marker""s Cartesian (x, y, and z) coordinates are listed per frame and no hierarchy or limb rotations are included.
Electromagnetic Trackers
Electromagnetic motion capture systems are part of the six degrees of freedom electromagnetic measurement systems"" family and consist of an array of receivers that measure their spatial relationship to a nearby transmitter. These receivers or sensors are placed on the body and are connected to an electronic control unit, in most cases by individual cables.
Also called magnetic trackers, these systems emerged from the technology used in military aircraft for helmet-mounted displays (HMDs). With HMDs, a pilot can acquire a target by locating it visually through a reticle located on the visor. A sensor on the helmet is used to track the pilot""s head position and orientation.
A typical magnetic tracker consists of a transmitter, 11 to 18 sensors, an electronic control unit, and software. A state-of-the-art magnetic tracker can have up to 90 sensors and is capable of capturing up to 144 samples per second. The cost ranges from $5,000 to $150,000, considerably less than optical systems. To take advantage of the real-time capabilities of a magnetic tracker, it must be connected to a powerful computer system that is capable of rendering a great number of polygons in real time. Depending on the needs of a particular project, the cost of this computer system alone could exceed the cost of the magnetic tracker.
The transmitter generates a low-frequency electromagnetic field that is detected by the receivers and input into an electronic control unit, where it is filtered and amplified. Then it is sent to a central computer, where the software resolves each sensor""s position in x, y, and z Cartesian coordinates and orientation (yaw, pitch, and roll). This data is piped into another algorithm that, in most cases, will convert each sensor""s global orientation and position into one hierarchical chain with only one position and multiple rotations.
The whole process is not truly real-time, but it is close, depending on the amount of filtering, amplifying, and post-processing, and the speed of the connection between the control unit and the host computer. Slow and congested Ethernet connections can slow this process down considerably. Magnetic trackers have a specification called latency, which indicates the amount of time elapsed between the data collection and the display of the resulting performance. This specification can vary from a few milliseconds to a few seconds.
Magnetic trackers such as the Flock of Birds by Ascension Technology Corporation use direct current (DC) electromagnetic fields, whereas others, such as the Polhemus ULTRATRAK PRO, use alternating current (AC) fields. Both of these technologies have different problems associated with metallic conductivity. AC trackers are very sensitive to aluminum, copper, and carbon steel, but not as sensitive to stainless steel or iron, whereas DC trackers have problems with ferrous metals, such as iron and steel, but not with aluminum and copper.
Many of these conductivity problems are caused by the induction of a current in the metal that creates a new electromagnetic field that interferes with the original field emitted by the tracker. These new fields are called eddy currents. Some magnetic trackers use special algorithms to compensate for these distortions by mapping the capture area, but these calibrations only work with static, predefined problem areas such as metallic structures in buildings. In most cases, it is better to avoid any high-conductivity metals near the capture area. This limitation makes the magnetic tracker difficult to transport to different locations.
Electromechanical Suits
The electromechanical motion capture suit is a group of structures linked by potentiometers or similar angular measurement devices located at the major human joint locations; it is driven by a human body""s actions.
Potentiometers are components that have been used for many years in the electronics industry, in applications such as volume controls on old radios. A slider moving along a resistor element in the potentiometer produces a variable voltage-potential reading, depending on what percentage of the total resistance is applied to the input voltage. The potentiometers used for motion capture suits and armatures are much more complex versions of the old radio volume knob; they are sometimes called analog or digital angular sensors.
One big drawback of electromechanical systems based on potentiometers is their inability to measure global translations. In most cases, an electromagnetic sensor is added to the configuration to solve this problem, but that subjects the setup to the same disadvantages as the electromagnetic systems, such as sensitivity to nearby metals. In addition, the design of most of these devices is based on the assumption that most human bones are connected by simple hinge joints, so they don""t account for nonstandard rotations that are common to human joints, such as in the shoulder complex or the lower arm.
There exists motion tracking systems which are based, at least in part, on radio frequency (RF). These systems typically employ the use of tags which serve as both receivers and transmitters. Sensors are placed around a capture area and such sensor also transmit and receive signals. The sensors transmit to the tags on the objects instructing them to transmit a signal in order to track each tag. It will be appreciated by those skilled in the art that the complexity and cost of the system is increased when the tags and sensors are transceivers. Additionally, such systems in the past have required time synchronization between the tags and the sensors, or at least between the multiple sensors. This even further complicates the system. Moreover, such radio frequency systems only have an accuracy within a few inches.
Accordingly, there is a need for a motion capture system utilizing higher frequencies to record much more motion data than existing motion capture systems. Such a system should be capable of capturing not only linear motion, but also rotational motion. Such a motion capture system should yield a three-dimensional representation of the motion which can be examined from any possible angle. Such a system should also be capable of being used in a variety of settings. Such a system should further be able to track thousands of objects, or points, to a very high degree of accuracy. The present invention fulfills these needs and provides other related advantages.
The motion tracking system of the present invention tracks the motion of human, animal or inanimate subjects using radio frequency technology. The present invention is an outside-in system. More specifically, the present invention employs multiple antennas and specialized software to locate transmitters placed on the objects. The global location of each transmitter is combined to create a digital representation of the tracked object. The data is also used to create movement-related databases, including, but not limited to, statistics on speed, velocity, and other types of motion analysis.
Applications of this invention include video games, television, cartoons, commercials, music videos, feature films, digital extras, digital stunts, and digital crowds. This invention provides the advantages of optical systems (extremely accurate, large number of markers, easy to change marker configuration, performers not constrained by cables, and large performance areas) without the disadvantages (extensive post-processing, expensive hardware, inability to capture occluded markers, and the need for a controlled environment).
The method for tracking object movement in a capture zone in accordance with the present invention generally comprises the steps of placing a plurality of sensors around the capture zone. Typically, at least four sensors are required. A stationery reference tag is placed within the capture zone. At least one tag, and preferably a plurality of tags, are coupled to the objects to be tracked in the capture zone. Signals are periodically transmitted from the reference tag and the object tags. These signals are received at the sensors where the code phase and carrier phase measurement for each signal received is extracted. The code phase and carrier phase measurements are processed to determine the position of the tags with respect to the reference tag.
The reference tag and each object tag transmits signal burst modulated with a digital data pattern having a portion common to all tags, and a portion unique to each tag. The common portion comprises a Neuman-Hofman synchronization code. The unique portion, comprising the identification code of the tag, comprises code words of a binary extended quadratic residue code. The multiple object tags are divided into segments of a transmission frequency and range. Preferably, the band range is at the 5.8 GHz band range, although other band ranges can be used, such as 60 GHz or 2.4 GHz.
The tag signals are circularly polarized. The processing of the code phase and carrier phase measurements of each tag includes the step of forming single difference measurements for each object tag measurement and reference tag measurement. Double difference measurements are then formed by pair wise differencing the single difference measurements from different sensors relating to the same object tag to determine the position of that tag. The double difference measurements are then combed.
The carrier phase measurements are corrected for sensor antenna phase center variations as a function of signal arrival angle. Typically, the sensors are calibrated to reduce error. The carrier phase measurements are also corrected for variations in the radio refractive index.
The change of object tag position over time can be used to determine velocity, acceleration and jerk of each object tag in any given axis. Such can be used by a biomechanical solver, or other software, to determine object movement, create animation, etc.
Other features and advantages of the present invention will become apparent from the following more detailed description, taken in conjunction with the accompanying drawings which illustrate, by way of example, the principles of the invention.